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Abstract 

APPARATUS AND METHOD FOR CAPTURING AND 
REPRODUCING A SERIES OF VIDEO FRAMES 

Disclosed is a video capture apparatus (101) for capturing a series of video frames. 
The video capture apparatus (101) comprises level of interest (LOI) input means (112) 
operable by a user of the video capture apparatus to generate a LOI signal (Fig. 3) 
indicative of the user's interest in a scene (124) being captured. The appartus (101) 
also includes LOI recording means for recording the LOI signal with the video frames 
being captured. 
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APPARATUS AND METHOD FOR CAPTURING AND 
REPRODUCING A SERIES OF VIDEO FRAMES 
Field of the Invention 

The present invention relates to a video capture apparatus, a video replay 
apparatus, and associated methods, for capturing and replaying a series of video frames 
having associated level of interest ("LOI") signals. The invention has been developed 
primarily for use with amateur video cameras and live video capture, and will be 
described hereinafter with reference to these applications. However, it will appreciated 
that the. invention is not limited to these fields of use. 

Background 

Since the introduction of home video cameras, amateur recording of various 
events has grown immensely in popularity. Video cameras themselves have decreased 
in size and cost, to a point where they are accessible to a majority of consumers. Such 
video cameras typically include "record", "play", "fast forward", "rewind" and "stop" 
buttons for controlling recording and playback of captured video frames. Various other 
features, such as date stamping of frames and automatic zoom, are usually included to 
improve the usability and desirability of the product. 

Unfortunately, when recording a particular event, it is difficult for a camera 
operator to know in advance when something of significance will happen. Typically, it 
is; not feasible to simply await the commencement of the event prior to starting 
recording, as, particularly with transient events, critical aspects will be missed as the 
camera initialises itself and begins the recording process. 

An alternative is to record relatively long sequences of video frames, which is 
time consuming and typically produces a large amount of relatively uninteresting 
footage. The resultant video is often a tedious recording of an event, the majority of 
the footage being of little interest, even to the author of the video. 

One solution to this problem is to edit out relatively uninteresting parts at a 
later date. However, this is a time consuming process which involves expensive 
equipment, and as a result, is an option only rarely undertaken by amateurs. Also, 
whilst the relatively uninteresting parts may not be of great interest for regular viewing, 
there may be some desire, from time to time, to see all information recorded at an 
event, including the relatively less interesting parts. For this reason, deletion of all but 
the most interesting points of a video recording is often unacceptable. 
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Where editing is acceptable, one way of improving the efficiency thereof is to 
locate "cuts" in the series of video frames, where one scene ends and another commences. 
However, whilst this may be of some help in locating the start of a particular scene, there 
are generally too many cuts for this method to useful, and the position of any relatively 
interesting parts of the video with respect to the cuts is not predictable. 

Summary of the Invention 

It is an object of the present invention to overcome or at least substantially 
ameliorate one or more of the disadvantages of the prior art. 

In accordance with a first aspect of the present invention there is provided a 
video capture apparatus for capturing a series of video frames, the video capture apparatus 
comprising: 

level of interest (LOI) input means operable by a user of the video capture 
apparatus to generate a LOI signal indicative of the user's interest in a scene being 
captured, said LOI signal having a range of more than two values; and 

LOI recording means for recording the LOI signal with each corresponding video 
frame being captured. 

Preferably, the LOI input means is configured to enable a user to select any one 
of a plurality of levels of interest for recording with the video frames being captured. 

In a preferred embodiment, the LOI input means is a key, button or switch 
mounted on an exterior casing of the video apparatus. Desirably, the LOI indicated by the 
LOI signal is related to pressure applied to the key or button by a user, and is preferably 
proportional to that pressure. Alternatively, the LOI input means is a slider or rotary dial, 
the LOI signal being generated on the basis of a selected position of the slider or rotary 
dial at a given point in time. 

In accordance with a second aspect of the present invention there is provided a 
video replay apparatus for replaying a series of video frames in accordance with a level of 
interest (LOI) signal associated with the video frames, the apparatus comprising: 

video replay means for reproducing the video frames from a video frame source; 

LOI extraction means for extracting said LOI signal associated with each frame 
in the series of video frames, said LOI signal having a range of more than two values; and 

replay control means for controlling the video replay means in reliance on the 
extracted LOI signal. 
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Preferably, the video replay apparatus includes LOI threshold selection means for 
setting a LOI threshold, the replay control means being configured to select frames having 
an associated LOI signal above or below the LOI threshold. 

Preferably, the video replay apparatus further includes LOI display means for 
5 displaying a summary of the LOI signals associated with sum or all of the series video 
frames. 

In accordance with a third aspect of the present invention there is provided a 
method of identifying a level of interest ("LOI") within a series of video frames, said 
method comprising the steps of: 
10 (a) capturing a video frame; 

(b) sampling an associated LOI value from LOI input means operable by a 
user, said LOI value having a range of more than two values; and 

(c) recording the captured video frame and associated LOI to a video 
storage medium, such that a series of the video frames can be reproduced from the video 

15 storage medium based on their associated LOI values. 

In accordance with a fourth aspect of the present invention there is provided a 
video capture apparatus for capturing a series of video frames, said video capture 
apparatus comprising: 

input means operable by a user of the video capture apparatus to generate a 
20 signal indicative of the user's level of interest in a scene being captured, said signal having 
a range of more than two values; and 

recording means for recording said signal and said video frames upon a common 
recording medium. 

Other aspects of the present invention are also disclosed. 
25 Brief Description of the Drawings 

A preferred embodiment of the invention will now be described, by way of 
example only, with reference to the accompanying drawings in which: 

Fig. 1 is a simplified perspective view of a video capture apparatus and a scene 
to be captured; 

30 Fig. 2 is a flow chart showing the basic steps in a method of recording a level of 

interest associated with one or more frames in a series during video capture thereof; and 

Fig. 3 is a display plotting a level of interest (LOI) with respect to time for a 
series of video frames recorded by the apparatus of Figure 1 . 
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Detailed Description including Best Mode 

Fig. 1 shows a video capture apparatus in the form of a video camera 101 for 
capturing a series of video frames. The video camera 101 includes an array of operating 
5 buttons 102, including a PLAY button 104, a STOP button 106, a FAST FORWARD 
button 108, a REWIND button 1 10, and a RECORD button 1 1 1. A level of interest (LOI) 
input means in the form of a LOI button 112 is also provided. The video camera 101 also 
includes standard features such as a viewfinder 114, a casing 116, a video cassette slot 
1 18 to accept a video cassette 120, and an objective lens 122. 
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A user (not shown, for clarity) points the video camera 101 generally towards a scene 
124, in this illustration formed of a person standing beside a tree. 

The basic steps involved in implementing a preferred embodiment of the 
invention can now be described with reference to Fig. 2. 

The user uses the video camera 101 to record, in this case, the scene 124. The 
record button 111 is depressed at step 201, this causing the video camera 101 to initiate 
recording of scene 122 at step 202. The procedure required to initiate recording in this 
fashion is well known by those in the art, and may include such substeps as unspooling 
tape from the video cassette 120 and wrapping it around a recording head (not shown 
but well known in the art), commencing rotation of the recording head and starting 
movement of the tape thereacross. A charge-coupled device (also not shown but 
similarly well known) or other transducer is then used to capture an image of the scene 
124 formed thereon by objective lens 122. 

The image passing through objective lens 122 is captured and formatted into a 
frame as indicated at step 203. Simultaneously or immediately thereafter, a LOI input 
selected by the user via the LOI button 112 is sampled at step 204, and recorded to the 
video cassette 118 in step 205 along with the formatted frame captured in step 203. 

The captured frame and associated sampled LOI value can be recorded to the 
video cassette 120 by any suitable means. In an analog format, such as VHS, the LOI 
associated with a particular frame can be recorded as an analog signal in the vertical 
blanking interval immediately before or after the associated frame, in a similar fashion 
to the way TELETEXT (trade mark) information is recorded. In this way, the tape can 
be replayed by a prior art video playback device. 

If the video frames are recorded via a digital method, the LOI signal can be 
recorded alongside each frame in a separate data field. Many digital formats, such as 
MPEG-4, provide private data fields, which can store the LOI signal for each frame. 

Once the video camera 101 is in operation, the user can input his or her 
subjective level of interest in a number of ways. For example, where the LOI button 
112 is formed by a simple switch, the level of interest may be varied by the rate at 
which the button is pressed. No pressing may be intepreted as a low level of interest, 
infrequent pressing as a moderate level of interest, and continuous pressing as a high 
level of interest. As a consequence a range of levels of interest may be recorded. 

Alternatively, the LOI button 112 may be pressure sensitive, such that an 
increase in pressure of depression is interpreted as a corresponding increase in the level 
of interest in the scene being recorded. Such an LOI signal is essentially continuous 



466637 CFP46637AU MMcdia16 



[I:\ELEC\CISRA\MMEDIA\MMEDIA 1 6J466637.doc: LDP 



(analog) and may be recorded as such in analog systems such as VHS. Alternatively, 
such a signal may be digitised prior to recording to give specific quantum levels of 
interest that may be discriminated against at some later time. 

Preferably, more than two levels of interest are are available to be recorded as 
such allows scope for the emotive and subjective response of the user of the camera 101 
and also subsequently for the audience of the recorded footage as interpreted by the 
LOI signal. 

The subjective LOI is recorded with the frames as they are captured by the 
video camera 101, the frames and level of interest signal being recorded to the video 
cassette 118 as described above. 

When the user wishes to finish recording, determined at step 206, the STOP 
button is pressed at step 207. 

The video camera 101 also represents an embodiment of a video replay 
apparatus for replaying a series of video frames in accordance with a LOI associated 
therewith, as discussed below. However, the invention can also be embodied in 
standard video cassette machines, optical disc players or any playback means suitable 
for the medium on which the frames are recorded. 

The LOI information can be used in a number of ways during playback of the 
video frames. In one embodiment, the speed of playback is a function of the LOI 
signal for the frame currently being replayed. For example, areas of the recorded tape 
flagged as being of a low interest are played back relatively quickly, whilst areas of 
relatively high interest are played back at normal speeds. 

To ensure that frames of relatively high interest are not missed, in one 
embodiment the LOI signals are read slightly ahead of the actual video frames being 
reproduced, such that the video replay speed slows down shortly before the actual area 
of high LOI is replayed. This forward reading can be achieved by several methods, 
depending upon the nature of the storage device. 

For example, in the analog embodiment, the LOI information is preferably 
stored in a different (advance) portion of the tape than the video frames to which the 
LOI relates, making it easier to detect the area of relatively high level of interest before 
it is reached. For example, the LOI information can be stored along an edge of the 
video tape, rather than being mixed with the video frames which necessitates recording 
using the rotary video head. A stationary head displaced from the rotary head can be 
used to read the LOI information from the edge of the tape for a particular frame before 
that frame is itself read by the rotary head. 
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In a digital device, various known digital delay or read forward methods and 
mechanisms can be used to introduce a lag between the LOI being read and the frame 
being reproduced. In the case of a random access digital medium such as a digital 
optical disc, hard disc drive, or random access memory, the requisite lag can be 
introduced by relatively simple software manipulation during playback. 

In an alternative embodiment, the LOI button 112 and RECORD button 111 
can be combined into a single combination key (not shown). Initially, depressing the 
combination key commences the recording process, whilst subsequent pressure applied 
by the user generates a LOI signal for recording with the captured video frames, as 
described in more detail above. In yet another embodiment, the LOI input means can 
take the form of a turnable knob or a slider, the relative position of which denotes the 
users level of interest at a given time. 

Turning to Fig. 3, there is shown a graphical summary of the LOI with respect 
to time for a particular series of video frames. First and second preselected thresholds 
Tl and T2 are shown on the vertical LOI axis, whilst corresponding subseries of 
frames PI and P2 are shown on the horizontal time/frames axis. 

This graphical summary of LOI information associated with the series of 
frames can be shown, for example, either on a blank screen or superimposed over 
video replay. A user can manually select a replay start point on the basis of the 
displayed LOI information, and replay controller within a video cassette player will 
either fast forward the cassette tape or select the appropriate frames from random 
access storage for reproduction on the display. 

As shown in Fig. 3, the user can select a LOI threshold, whereby a replay 
controller controls the replay apparatus such that only video frames having an LOI 
above the selected threshold are replayed. For example, selecting a relatively high 
threshold T2 results in replay of a relatively small sub-series PI of the series of video 
frames. Selecting a lower LOI threshold Tl results in replay of a larger subseries P2 of 
the series of video frames. A user can experiment with different threshold levels, until 
a satisfactory compromise between the number of frames selected and their subjective 
levels of interest is reached. 

The LOI signals associated with the video frames can also be used to 
automatically produce a printout of relatively interesting parts of the tape. A method 
similar to that described above can be used to select a subseries of relatively interesting 
frames. Alternatively, peaks in the LOI signals are used to indicate frames of relatively 
high interest. A number of frames are selected from the subseries for storage or 
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reproduction. The selected frames can then form the basis of a summary page recorded 
at the start of the series of frames to be replayed. 

The selected frames can be printed, optionally in "thumbnail" format, for 
storage with the cassette tape or other storage medium as a reminder of their content. 
In yet another embodiment, the selected frames are reprinted using a relatively high 
quality printer, and used to create the equivalent if a still-image camera photo album. 
Also, the selected frames can be stored on a computer or the like, and used to form a 
"virtual album" to be viewed on a computer monitor. 

For a given LOI threshold and its associated subseries of video frames, a 
search can be made to select particular frames for use in still-image albums or contents 
pages as discussed in previous claims. In one embodiment, a search of the frames 
having an associated LOI above the selected threshold is automatically made on the 
basis of image content. Preferably, the search is based on an edge detection or 
sharpness assessment of the content of the video frames, which results in a relatively 
high probability of automatic selection of a frame of correspondingly high interest. 

Whilst the embodiments so far described have considered the use of a variable 
LOI level, the LOI button can also be implemented in a "binary" mode, in which only a 
"yes" or "no" LOI signal is recorded. In such embodiments, the camera is used to 
mimic the function of a still image camera whilst continuous recording is underway. 
During recording, the binary LOI button is selectively depressed to indicate a scene of 
particular interest, a "yes" LOI signal being recorded alongside the frame being 
captured. Frames captured when the LOI button is not depressed either have no LOI 
signal recorded with them, or an explicit "no" LOI signal. The frames which have 
been annotated with a "yes" can be reproduced as either a thumbnail summary or a 
series of images representing the highlights of the series of video frames, as described 
in earlier claims. 

In an alternative embodiment, the LOI signal for each frame can be measured 
and stored automatically, without the user having to consciously vary pressure on a 
button or key, or set a dial or slider. One way of achieving this is to use a bio-sensor 
to measure involuntary physical responses and generate a LOI signal based on the 
instantaneous relative output of the bio-sensor. The bio-sensor can be any suitable 
type, and can measure such quantities as: 

skin impedance; 

pulse rate; 

eye position and movement; 
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pupil dilation; 

eyebrow position and movement; and 
mouth shape. 

In most of these cases, the basic bio-sensor output will vary from user to user, 
and so typically a relative measurement will need to be made. Accordingly, it is 
necessary for the bio-sensor and associated LOI signal generation means to " learn " a 
user's basic physical responses over time such that a relative LOI signal is produced in 
accordance with changes in the user's physical responses. 

Although the invention has been described with reference to a number of 
specific examples, it will be appreciated by those skilled in the art that the invention 
may be embodied in many other forms. 

In the context of this specification, the word "comprising" means "including 
principally but not necessarily solely" or "having" or "including" and not "consisting 
only of. Variations of the word comprising, such as "comprise" and "comprises" 
have corresponding meanings. 
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The claims defining the invention are as follows: 



• • 

• • 



1. A video capture apparatus for capturing a series of video frames, said video 
capture apparatus comprising: 

level of interest (LOI) input means operable by a user of the video capture 
apparatus to generate a LOI signal indicative of the user's interest in a scene being 
captured, said LOI signal having a range of more than two values; and 

LOI recording means for recording the LOI signal with each corresponding video 
frame being captured. 

2. A video capture apparatus according to claim 1, wherein the LOI input means is 
configured to enable a user to select any one of a plurality of levels of interest for 
recording with the video frames being captured. 



is 3. A video capture apparatus according to claim 1 or 2, wherein the LOI input 
means comprises an actuator mounted on an exterior of the video capture apparatus. 



10 



20 



4. A video capture apparatus according to claim 3, wherein the LOI indicated by the 
LOI signal is related to pressure applied to the actuator by a user. 

5. A video capture apparatus according to claim 4, wherein the LOI indicated by the 
LOI signal is proportional to the pressure applied to the actuator by a user. 



25 



30 



6. A video capture apparatus according to claim 1 or 2, wherein the LOI input 
means comprises a slider or rotary dial, the LOI signal being generated on the basis of a 
selected position of the slider or rotary dial at a given point in time. 

7. A video capture apparatus according to claim 1 or 2, wherein the LOI input 
means comprises one or more bio-sensors, a relative output of the bio-sensors being 
interpreted to provide said LOI signal. 



8. A video capture apparatus according to claim 7, wherein the bio-sensors measure 
one or more of the user's skin impedence, pulse rate, eye position, pupil dilation, eyebrow 
position, and mouth shape. 



466637.doc 



-10- 



9. A video replay apparatus for replaying a series of video frames in accordance 
with a level of interest (LOI) signal associated therewith, said apparatus comprising: 

video replay means for reproducing video frames from a video frame source; 
LOI extraction means for extracting a LOI signal associated with each frame in 
5 the series of video frames, said LOI signal having a range of more than two values; and 

replay control means for controlling the video replay means in reliance on the 
extracted LOI signal. 

10. A video replay apparatus according to claim 9, further comprising LOI threshold 
10 selection means for setting a LOI threshold, said replay control means being configured to 

select frames having an associated LOI signal above or below the LOI threshold. 



• 

* 

• • • 



11. A video replay apparatus according to claim 9 or 10, further comprising LOI 
display means for displaying a summary of the LOI signals associated with some or all of 

is the series of video frames. 

12. A video replay apparatus according to claim 11, wherein the summary of LOI 
signals is graphically displayed on a display monitor. 



20 13. A method of identifying a level of interest ("LOI") within a series of video 
frames, said method comprising the steps of: 

(a) capturing a video frame; 

(b) sampling an associated LOI value from LOI input means operable by a 
user, said LOI value having a range of more than two values; and 

25 (c) recording the captured video frame and associated LOI to a video 

storage medium, such that a series of the video frames can be reproduced from the video 
storage medium based on their associated LOI values. 



30 




14. A method according to claim 1 3 wherein steps (b) precedes step (a). 

15. A method according to claim 13 wherein steps (a) and (b) occur simultaneously. 

16. A video capture apparatus according to any one of claims 1 to 8 wherein said 
video frames are recorded upon a recording medium and said LOI signal is recorded upon 
said recording medium. 

466637.doc 
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17. A video capture apparatus for capturing a series of video frames, said video 
capture apparatus comprising: 

input means operable by a user of the video capture apparatus to generate a 
5 signal indicative of the user's level of interest in a scene being captured, said signal having 
a range of more than two values; and 

recording means for recording said signal and said video frames upon a common 
recording medium. 

10 18. A video capture apparatus according to claim 17 wherein said signal is formed 
in a manner at least partly dependent upon an emotive response of the user to the scene 
being captured. 

19. A video capture apparatus according to claim 17, wherein said input means is 
is configured to generate a substantially continuous intermediate signal that is modified to 

form said signal. 

20. A video capture apparatus substantially as described herein with reference to Fig. 
1 of the drawings. 

20 

21. A method of identifying a level of interest ( H LOF) within a series of video 
frames substantially as described herein with reference to Fig. 2 of the drawings. 

22. A video recording system substantially as described herein with reference to the 
25 drawings. 

23. A video reproduction system substantially as described herein with reference to 
the drawings. 



30 DATED this twenty first Day of November 2001 

Canon Kabushiki Kaisha 
Patent Attorneys for the Applicant 
SPRUSON & FERGUSON 
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